Using 3-Dimensional Meshes To Combine Image-Based and Geometry-Based Constraints

A uni ed framework for 3{D shape reconstruction allows us to combine image-based and geometry-based information sources. The image information is akin to stereo and shape-fromshading, while the geometric information may be provided in the form of 3{D points, 3{D features or 2{D silhouettes. A formal integration framework is critical in recovering complicated surfaces because the information from a single source is often insu cient to provide a unique answer. Our approach to shape recovery is to deform a generic object-centered 3{D representation of the surface so as to minimize an objective function. This objective function is a weighted sum of the contributions of the various information sources. We describe these various terms individually, our weighting scheme, and our optimization method. Finally, we present results on a number of di cult images of real scenes for which a single source of information would have proved insu cient.